The statements in this section merely provide background information related to the present disclosure and may not constitute prior art.
Travelers in a foreign land often need to be able to read and understand some text written in a foreign language, such as a restaurant name or address, a restaurant menu, a street sign, a book, a map, a train schedule, or a newspaper. Conventionally a traveler may use a foreign translation book, hire a guide, or ask local people for help. These approaches are awkward and the increasing use of English by foreigners throughout the world as their second language is not going to end this language barrier.
Translating devices are known that use complex image processing and optical character recognition (OCR) software. OCR has significantly improved since its inception in the early 1990s and it is used on the Internet; however, foreign travelers generally do not have a mobile device with an Internet connection in a foreign country. Thus a translation device for a traveler needs to function adequately offline, that is, without the resources afforded by a connection to the Internet and access to an online server.
Offline OCR applications for mobile camera devices have size limitations in terms of size of the program code. There are limits to the speed of the image processing and OCR algorithms offline as well. There are limitations in types of processors and in memory resources in mobile camera devices. Offline mobile translator devices also suffer from a lack of translation accuracy and reproducibility. Generally mobile translation devices will be used to capture a single image frame of the foreign text to be translated. OCR will be performed on the captured image frame of the foreign text to translate the foreign language text into a language that can be read by the traveler. However, during image capture of the foreign text using a hand-held mobile camera device such as a smart phone, there are image capture problems which include camera movement, poor text image focus, and improper foreign text illumination. OCR requires a clear distinctive text image for an accurate and stable foreign text translation so a non-clear text image will mislead the OCR software, which will then produce a defective language translation. Thus it is known that offline translation apps for mobile camera devices such as smartphones frequently do not perform accurate and stable translations. The translations may fluctuate, jitter, or even make no sense at all.
For these reasons, there exists an important need for solutions to these problems related to current translation technology for mobile camera devices to bring improved speed, accuracy, and meaning in translations. There is a need for translations in real-time and with grammar linguistics to allow for a better touristic experience in a foreign land. What are needed are a method, system, and apparatus for rapid and meaningful translation of a foreign language text in real-time, on a resource-constrained mobile device, without the requirement for Internet connectivity.
Therefore, it would be an advancement in the state of the art to provide a method for rapid and accurate translation of a foreign language in real-time with accuracy to resolve the shortcomings of existing solutions. It would also be an advance in the state of the art to provide this translation method in a mobile device that can translate the foreign language in real-time without the need for Internet connectivity to automatically provide the tourist with meaningful information. It would be a further an advancement that such a translation is cost-efficient, does not require translators or dictionaries, or manual entering of text into the mobile device. It would be a further advancement in the state of the art to provide multi-line translation methods and systems that can translate multiple lines of foreign languages, such as, but not limited to, multiple lines of a foreign language menu. Finally, it would be a further advancement in the state of the art to provide user interfaces, systems, and methods for translating character sets that are displayed vertically. It is against this background that various embodiments of the present invention were developed.